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(54) System and method for scaling video 

(57) Scaling of video is performed using area 
weighted averaging of input pixels to calculate coeffi- 
cients to multiply with luminescence and crominence of 
input pixels. Such coefficients are produced for both the 
vertical and horizontal scaling directions of the input vid- 
eo stream. When scaling down or scaling up, scaling is 
first performed in the vertical direction to produce par- 
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tially scaled pixels, which are then utilized for scaling in 
the horizontal direction. When scaling up, a pre-interpo- 
lation or pre-replication process is utilized to double the 
inputted pixel grid which doubled pixel grid is then uti- 
lized to scale down to the desired pixel grid size, which 
is greater than the originally inputted pixel grid size. 
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Description 

TECHNICAL FIELD OF THE INVENTION 

This invention relates in general to data processing systems, and, in particular, to scaling of video in a data process- 
ing system. 

BACKGROUND OF THE INVENTION 

A rapidly emerging multi-media application in the personal computer arena is the joint display of full-motion video 
along with conventional graphic applications. For example, a user may wish to display a window of video (from a VCR 
or camera) along with a window containing a spreadsheet application. Users of currently available graphical-user in- 
terface (GUI) operating systems expect to be able to re-size the video window to any arbitrary size, and expect the 
horizontal and vertical dimensions of the window to be fully independent. They may want the window to be very small, 
so that they can glance at it while working on another application, or they may want it to be full-screen size so that it 
is the focus of their attention. 

The source of this video may be a "live" image from a device such as a camera or VCR, or it may be a decompressed 
image from a software or hardware decompressor. In any case, the video source will be a fixed size (160 horizontal 
pixels by 120 vertical pixels, for example). For the reasons indicated above, the user will want to be able to re-size 
(scale) the video stream to match the desired window size on the display (640 horizontal pixels by 480 vertical pixels, 
for example). The input stream may be smaller than the desired output size (scale-up needed), or it may be larger than 
the desired output size (scale-down needed). This need to be able to scale a video stream to an arbitrary larger or 
smaller size is the problem that this invention solves. 

SUMMARY OF THE INVENTION 

It is an objective of the present invention to provide an improved system and method for scaling a video image by 
generating coefficients in both the vertical and horizontal directions as a function of an area weighted average of the 
contributions of input pixels to output pixels in both the vertical and horizontal directions and as a function of a scaling 
ratio of input pixels to output pixels in both the vertical and horizontal directions, then multiplying inputted pixels of a 
video image with the coefficients generated in the vertical direction as a function of an area weighted average of 
contributions of input pixels to output pixels in the vertical direction and this multiplying step resulting in vertically scaled 
pixels, and then multiplying these vertically scaled pixels by the coefficients generated in the horizontal direction re- 
sulting in vertically and horizontally scaled pixels. 

The present invention performs a scaling up function by essentially logically doubling the inputted pixel image 
through either interpolation or duplication and then scaling down this logically sca!ed-up image to the desired pixel grid 
size using the above described technique. 

The present invention preferably performs the scaling down and scaling up functions in both the vertical and hor- 
izontal directions independently. 

The present invention also provides a system for scaling up a video image from an m x n pixel grid to an (m+b) x 
n pixel grid, wherein b is an integer greater than zero and less than or equal to m, said system comprising: 

means for logically scaling up said video image from an m x n pixel grid to an (m+c) x n pixel grid, wherein c is an 
integer greater than zero, and wherein c is greater than b; and 

means for scaling down said (m+c) x n pixel grid to said (m+b) x n pixel grid. 

Furthermore, said means for scaling up said video image from an m x n pixel grid to an (m+c) x n pixel grid further 
comprises means for interpolating or replicating pixels along the m axis. 

A preferred embodiment of the present invention is implemented in a data processing system including a processor, 
a storage device, and an input/output means including a display device. 

BRIEF DESCRIPTION OF THE DRAWING 

For a more complete understanding of the present invention, and the advantages thereof, reference is now made 
to the following descriptions taken in conjunction with the accompanying drawings, in which: 

FIGURE 1 illustrates a representational scheme for scaling down a ratio or 2:1 ; 
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FIGURE 2 illustrates a representational scheme for scaling down 4:3; 
FIGURE 3 illustrates a representational scheme for scaling down 4:3 in one dimension; 
FIGURE 4 illustrates a logical block diagram of a Z-Engine; 
FIGURE 5 illustrates various scale down pixel overlay cases; 
FIGURE 6 illustrates a hardware configuration for scaling down video; 
FIGURE 7 illustrates pre-replicated and pre-interpolated pixel streams; 
FIGURE 8 illustrates logical pixel coefficients for pre-replication and pre-interpolation; 
« FIGURE 9 illustrates overlap of logical pixels In pre-interpolation; 

FIGURE 10 illustrates coefficient generation with two chained Z-Engines; 
FIGURE 11 Illustrates pre-replication overlay cases In one dimension; 
FIGURE 12 illustrates pre-interpolation overlay cases on one dimension; 
FIGURE 13 illustrates a hardware configuration for coefficient interpolation/reduction; 
» FIGURE 1 4 illustrates a hardware configuration for vertical accumulation; 

FIGURE 15 illustrates a hardware configuration for horizontal accumulation; 

FIGURE 16 illustrates a hardware configuration of converged scaling data flow for scale down and scale up; and 

30 

FIGURE 17 illustrates a data processing system for implementing the present invention. 
DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT OF THE INVENTION 

35 In the following description, numerous specific details are set forth such as specific word or byte lengths, etc. which 

provide a thorough understanding of the present invention. However, it will be obvious to those skilled in the art that 
the present invention may be practiced without such specific details. In other instances, well-known circuits have been 
shown in block diagram form in order not to obscure the present invention in unnecessary detail. For the most part, 
details concerning timing considerations and the like have been omitted inasmuch as such details are not necessary 

*o to obtain a complete understanding of the present invention and are within the skills of persons of ordinary skill in the 
relevant art 

Area Weighted Averaging 

& In a video scaling down process, an array of pixels is mapped onto a smaller array. The idea of a weighted average 

scaling scheme is for the output image to contain a good representation of the original (input) image, and to do so, the 
luminance and chromaticity of each area in the input image is represented in the same proportion in the output image. 
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Weighted Average 

Equation 1 shows the weighted average of two values, X, and X 2 . X^s coefficient, or weight, is w,; X^s coefficient 
is w 2 . The weighted average of these two variables is their weighted sum divided by the sum of the weights. 

w* *X* + wyX> 

VWU-V-! M (1) 



And in general, for n values 



WA = -=r LJ (2) 
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This general framework can be applied to scaling pixel images. In Area Weighted Averaging a scaled (output) pixel 
is formed by averaging input pixels, each weighted by the area of the input pixel which contributes to the output pixel. 
Given a regular m x m pixel grid Q, overlay annxn pixel grid P where n < m. For each pixel Oy in the n x n array 
compute a weighted average of the underlying pixels P,from the m x m array. The weighting coefficients of the calcu- 
s lation are proportional to the area of the m x m pixel grid overlaid by the pixel from the n x n grid. 
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10 For this discussion, assume a gray level image with each pixel in the range 0 - R (e.g., 0-255). The algorithm and 
the present invention is easily generalized in color. The algorithm manipulates the luminance of pixels. For now, assume 
isotropic scaling; later it will be generalized to non-isotropic scaling. Scaling will be in fixed integer ratios. 



Image Scaling with Area Weighed Averaging 

Referring to FIGURE 1, shown is an example wherein in scaling down 2:1 each output pixel Q t Is formed by 
averaging 4 input pixels (P v P 2 , P 3 > P 4 ). 

P 1 + P 9 + P. + P A 
^1= 1 % 3 4 (4) 

In this case all input pixels are weighted identically. When the scale factors are not evenly divisible, weighting 
coefficients vary, being proportional to the area of the input pixel that contributes to an output pixel. 

FIGURE 2 shows 4x4 input pixel array 201 and 3x3 output pixel array 202. In this example, 4x4 pixel array 
201 is being scaled down to 3 x 3 pixel array 202. Graphic 203 shows the way this problem is visually represented. 
Here, 4x4 grid 201 is overlayed by 3 x 3 grid 202, with alternate pixels shaded in the output array 202. In this repre- 
sentation, the sizes of the output pixels look larger than the sizes of the input pixels, but this is an artifact of the 
representational scheme. This representation is used to clearly show how more than one input pixel contributes to the 
information in one output pixel. 

This weighted average is shown in the first tine of the next set of equations. Q 1 is the weighted average of the 
contributing input pixels, normalized so that the output pixel luminances are in the same range (fr), as the input pixel 
luminances. The coefficients in the numerator are proportional to the area. The denominator is a constant which nor- 
malizes the sum. The following equations convert the fractional area weights (e.g., 1/2, 1/4) into a form with lower 
computational complexity. It is this form of the weighted average calculation that will be used in later discussions. The 
weighting coefficients, of course, depend on the scaling factor. 



Scaling in One Dimension 



3 3 5 9 6 



16 Ft 16 2 16 5 16 F< 

10 

so In FIGURE 3, there is shown a one-dimensional mapping of four pixels to three. Each output pixel, thus, iscomputed 
by calculating the weighting coefficients of the contributing input pixels. These coefficients can be derived from a Linear 
Congruent Sequence. Equatbn 6 shows the general form of the Linear Congruent Sequence. 

X /+1 = (a>X f + b) mod c (6) 
This representation, however, by definition restricts the range of the values from 0 to c - 1 . To change the range 
55 of the values, a variation on this sequence has been developed, so that the values can range from 1 to m. In this 
application, m and n are the scale factors. In scaling down an image, scale from m to n (e.g., from 4 to 3). 

Z M = (Z f b) mod c (7) 
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where 

b ~ (n) and c~ m 

and 

This equation generates a series of terms from which can be derived the weighting coefficients for each input pixel 
This form of the Linear Congruent Sequence is calied the Z sequence. The Z sequence is a function of m and n 
Consider, for example, the Z sequence for scaling from m = 4 to n = 3: 

Z(4:3) = 41 2341 234... 
10 from which the following coefficient pair sequence can be generated: 

(3,0 1,22,1 3,0 3,0 1,2 2,1 3,0 3,0 ...). 
The minimum non-repeating length of the Z sequence is given by 

L = LC^M ... 
n 1°) 
1S wherein LCM (m,n) is the least common multiple of m and n. 

Weighting coefficients can be directly calculated from the Z values by the following: 
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!Z f fi or Z, < n 
n for Z, * n 



CI -P , ' " <*> 



02 - n - C1 (9b) 
The algorithm uses the Z value as a decision variable in generating the weighting coefficient pair a C2 It will 
also be necessary to know how the input pixel is aligned with the output pixel(s) whether this pixel begins a new 
2S output pixel (PB), ends an output pixel (P£), or is split between two output pixels (PS) finishing one output pixel and 
beginning another. These flags are also derived from the Zdecision variable. These flags are used to steer the weighting 
calculation. The Z algorithm therefore supplies all the information needed to perform the area weighted averaqinq 
calculation. * y 



PB - Z f = m 

PS - Z f - n < 0 (10) 
PE - Z, - n = 0 

For an overview of the implementation of the Z-Engine, refer to FIGURE 4. The inputs to the Z-Engine are the 
required scale factor and the previous Z decision variable Zin. The outputs of the Z-Engine are two weighting coefficients 
(Q and C2), three pixel alignment flags (PB, PS, PE), and the next Zdecision variable Zout. The Z-Engine implements 
Equations 7, 9a, 9b and 1 0. 

Application To Rasterized Video 

The Area Weighted Averaging Calculation 

In a real-time sampled environment, the form of Equation 3 is not appropriate. In Equation 3, a single output pixel 
Q } is computed as a function of multiple input pixels, P, In the rasterized environment, however, the multiple Ps are 
not available simultaneously. Because the information is scanned, line by line, the information is available as a con- 
tinuous string of Ps. In the sampled environment, the algorithm must handle each input pixel, P, once and only once. 
Thus, instead, each input pixel's contributions to those output pixels it influences are calculated. 

Referring next to FIGURE 5, in one dimension, when scaling down, there are four ways for a given input pixel to 
map onto the output pixel stream. An input pixel can begin a new output pixel (/), contribute to a pending output pixel 
(//), finish an output pixel (///), or can be split between two output pixels, finishing one and beginning a second (/v). In 
addition, in the pass through case (scaling 100%) each input pixel is mapped 1 :1 to each output pixel (v). 

The Z-Engine generates two coefficients (CI ,C2); the weighting calculation generates two partial pixel weiqhtinqs 
(K9./C1 0) from these coefficients and the input pixel. 

[K1K2) = [C1C2)xP i 

Since at most one input pixel can contribute to two output pixels (finishing one and beginning another), the following 
accumulation algorithm (Equations 12-15) requires a single pending accumulator (A) and a single result accumulator 
(Q'). The one-dimensional accumulation algorithm is steered by the pixel alignment flags (PB, PS, PE). A valid com- 
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pteted pixel is indicated by the vaWflag. 



A if PB (12) 
otherwise 



A = 



K2 
Kl 
Kl 



+ A 



if PS 
else if 
otherwise 



(13) 



Valid = PS iPE 



(14) 
(15) 

PS I P£ is the logical or of the pixel split fP5/ and pixel end (PE) flags. N is the factor to normalize the result to the 
Input range. Q' is the result accumulator (before normalization) and A is the pending accumulator. PB Is the logical not 
of the pixel begin flag (PB). 

Extension to Ttoo Dimensions 



While it is possible to generate weighting coefficients and perform the weighting computation In two dimensions 
directly, when the algorithm is later extended to scaling up it is more compact to perform the scaling in each dimension 
separately in two stages. Pixels are first processed vertically, then the vertically weighted pixels are passed on to be 
scaled horizontally. 

For each dimension a separate Z-Engine is needed. The horizontal Z-Engine wiil Iterate for each pixel on a line, 
resetting at horizontal retrace. The vertical Z-Engine will iterate for each line of a field, resetting at vertical retrace. The 
two Z-Engines will generate separate weighting coefficients (CI 0 C2^and C2 H ) and separate pixel alignment 
flags (PB Vt PSyi PE V and PB H , PS^ PErf. Note that for all pixels on a given input line, the vertical coefficients and 
flags are constant. : 

To perform the vertical computation it is necessary to save and accumulate partial pixel data for each pixel of an 
input line, requiring a one line cache as wide as the widest input pixel line. The cache is a vertical accumulator for each 
input pixel. The vertical section performs the vertical weighting calculation (Equation 11 ), retrieves pending partial pixel 
data from the cache (fl), stores updated partial pixel data back to the cache (W), and passes finished (vertically) pixels 
(P v ) on to the horizontal section. Note that for input lines which are split between two output lines (PS V ) the cache is 
both read and written for each input pixel on that line. 

IK9K10] = [C1 v C2 v ]xP, 



-I 



K9 
K9 



+ R if PB, 



v 

otherwise 



(16) 



W 



K10 
K9 + A 
K9 



if PSy 

else if PB V 
otherwise 



(17) 



Valid v = PS V \PE V 



(18) 

The horizontal section accepts completed pixels from the vertical section (P v ), performs the weighting calculation, 
and outputs valid normalized, completed pixels. 

{K1K2} = [C1 H C2 H )xP v (19) 
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Kl + 
Kl 



A \iW B 
otherwise 



(20) 



A = 



K2 
Kl 
Kl 



+ A 



else if PB B 



(21) 



otherwise 




(22) 



Valtd Q = Valid v & Valid H = (PS V I PE V ) & (PS H I PE H ) 



(23) 



FIGURE 6 shows an overview of the Scale-Down architecture. This diagram shows the data flow for the algorithm 
for scaling down. Inputs are the horizontal and vertical scale factors (mH, nH, mV, nV) and the pixel input stream PL 
For each line of the input Pi , vertical coefficient generator 601 generates two vertical weighting coefficients (C1 V and 
C2y) and the relevant vertical pixel alignment flags as described with respect to the Z-Engine illustrated in FIGURE 4. 
For each pixel on this line, horizontal coefficient generator 602 generates two horizontal weighting coefficients (C1 H 
and C2 H ) and the relevant horizontal pixel alignment flags as described with respect to the Z-Engine illustrated in 
FIGURE 4. The incoming pixel P/is first multiplied in vertical multiply circuit 603 by the two vertical weighting coefficients 
to generate two partial pixels K9, K10 (Equation 11 ). Vertical accumulator 604, steered by the alignment flags combines 
these with pending partial pixel data from cache 605 (if any) and then updates the cache (if necessary) with pending 
data for the next line and passes the vertically completed pixel (PV) data to the horizontal section (Equations 16-18). 
This data is then multiplied in horizontal multiply circuit 606 by the two horizontal weighting coefficients to generate 
two more partial pixels Kl, K2 (Equation 19). Horizontal accumulator 607, steered by the horizontal alignment flags, 
combines these with its pending partial pixel data (if any), stores pending data (if necessary) and when finished, outputs 
a completed output pixel Q (Equations 20-23). 

Scaling Up 

Scaling Up Strategies 

Two strategies are employed for scaling up based on the idea of first doubling the input pixel stream to 200% and 
then applying the scale down algorithm. This pre-doubling can be via either simple replication or linear interpolation. 
Actually doubling the data stream (in two dimensions) would require four times as many computations; instead we 
logically double the data building the pixel interpolation or replication into the coefficient generation. 

We distinguish between logical input pixels which are never actually generated in the algorithm and the real input 
pixels which are operated on by the algorithm. 

For example, (in one dimension) scaling up to 1 60% would be equivalent to doubling 2 pixels to 4 and then scaling 
the 4 intermediate pixels to 3, which is referred to as 2:4:3 scaling. Equations 24a-c show the weighting calculation of 
scaJing the pre-interpolated pixels; Equations 25a-c show the weighting calculation after moving the interpolation into 
the coefficients. 




(25a) 



(24b) 



(24a) 



(24c) 
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(25b) 



0 3-f ,P 2 + f' P 3 



(25c) 



Generating Coefficients 

The scaling down algorithm examines one input pixel at a time. For scaling up with pre-replication (in one dimen- 
sion) we must evaluate two logical pixels for each input pixel; the actual input pixel and the replicated pixel. For scaling 
up with pre-interpolation we must evaluate three logical pixels for each input pixel; the actual input pixel and the two 
adjacent interpolated pixels. FIGURE 7 shows logical pre-interpolated and pre-replicated pixel streams. 

The Z algorithm generates two coefficients for each logical input pixel. For pre-replication, with two logical pixels 
this is a total of four coefficients for each Input pixel. For pre-interpolation this is a total of six coefficients. FIGURE 8 
shows a naming convention for these coefficients, which is utilized hereinafter. 

Whereas for scale down and scale up with pre-replication there is no Information overlap between logical pixels, 
with pre-interpolation each logical interpolated pixel overlaps with the next interpolated pixel (FIGURE 9). Note that 
the coefficients e and fare identical/ to the previous pixel's cand d coefficients. 

Referring to circuitry 1 003, 1 004 in FIGURE 1 0, to generate the coefficients for pre-replication and pre-interpolation 
two chained Z-Engines 1001, 1002 (similar to the one in FIGURE 4) are used which together iterate once for each 
input pixel. The two Z-Engines generate two coefficients each (a,6and c t d). In addition, the previous c,d values are 
saved as the next pixel's e, f coefficients. The inputs on the left are the scale factor term n, the Z 0 initial Zvalue and 
the inherited Zvalue of the previous pixel (Z M ). When scaling down, Z-Engine 1002 and the entire lower section is 
not utilized. When scaling up, the output Zvalue of Z-Engine 1001 is passed to Z-Engine 1002; the output Zvatue of 
Z-Engine 1 002 is the value used as Z M for the next pixel. Each Z-Engine 1 001 , 1 002 generates a pair of coefficients 
and a set of alignment flags. In addition, there are latches to copy c,d and PS 2 to e, f and PS 3 

Z-Engine 1001 generates the coefficients a,b and the flags PB P PS 1t PE V Z-Engine 1002 generates the coeffi- 
cients c,d and the flags PB 2 ,PS 2 ,PE 2 . For pre-interpolation the coefficients e, fare inherited from the previous pixel's 
c,d coefficients and PS 3 is inherited from the previous pixel's PS 2 flag. (For pre-interpolation PE 3 is not required and 
the PB 3 is trivially equal to PB^) 

Referring to FIGURE 11, in one dimension, when scaling up via pre-replication, there are six ways for a given input 
pixel to map onto the output pixel stream. An input pixel can map to a completed output pixel and begin a second 
pending output pixel (i); finish a pending output pixel and begin a second pending pixel (ii); and (Hi); finish a pending 
output pixel and complete a second pixel (iv); finish a pending output pixel, map to a second completed pixel and begin 
a third pending pixel (v); and be mapped to two output pixels in the 200% case (vi). 

Referring to FIGURE 12, in one dimension, when scaling up via pre-interpoiatlon, there are eight ways for a given 
input pixel to map onto the output pixel stream. An input pixel can finish an output pixel, and begin two pending pixels 
(i), (iii) and (iv); finish an output pixel and begin one pending output pixel (ii); finish two pending output pixels and begin 
one pending output pixel (v); finish two pending output pixels and begin two pending output pixels (vi); and finish one 
pending pixel, complete a second and begin a third pending pixel in the 200% case (vii). In addition, to these basic 
eight cases, there are two additional boundary conditions at the edge when the previous interpolated pixel is undefined 
(viii)and(ix). 

Coefficient Reduction 

Rather than employ four or six coefficients in the weighting calculation we interpose a coefficient reduction step 
between the logical coefficients and the weighting calculation. This step reduces the four coefficients in pre-replication 
to three and reduces the six coefficients in pre-interpolation to four. Coefficient reduction combines associated logical 
coefficients by summing together those logical coefficients which contribute to a common output pixel. For example, 
in pre-replication overlay case (i): c 1 - a + c. The pixel alignment flags are used to determine the pixel boundaries and 
thus, which coefficients to sum. 




a if PS t \PE t 

a+(fc+c) otherwise 



(26) 



8 



EP 0 710 925 A2 



15 



20 



25 



30 



40 



45 



50 



C2 = 




if (PS t & PS 2 ) (27) 
otherwise 



C3 \d if W&PSj (28) 

CJ \0 otherwise 



The Pre-Repllcatlon Calculation 

In Pre-Replication an input pbcel can contribute to at most three output pixels, only one of which can be pending 
from a previous pixel. Therefore as in the scaling down scenario, a single pending accumulator (A) is needed. Instead 
of one result accumulator though, two are needed {Q' A and Q q). 

Each pixel is weighted by three coefficients to form three partial pixel terms: 

[K1K2K3) = [C1C2C3}'Pi (29) 

If there is a pending partial pixel (if PB 1 is FALSE), then it is added to K1 to form the first result accumulator {Q A ): 



Kl + A i£PB t (30) 
Kl otherwise 



The second result accumulator (Q ' B ) is unconditionally loaded with K2: 

Q' 0 =K2 (31) 

The pending accumulator is loaded with either K2 or K3 depending on whether the replicated pixel contributes to 
two or three output pixels: 



■{ 



K3 if (PS X | PE X ) & (PS Z | PEJ (32) 

K2 otherwise 



55 The contents of the first result accumulator (Q ' A ) is always valid (at least one pixel is output for each input pixel 

when scaling up): . 

Valid A = 1 (33) 

And, the result accumulator (Oy is valid if: 

Valid B = (PS 1 \PE 1 )&(PS 2 \PE 2 ) (34) 

Equations 29-34 illustrate the pre-replication accumulation algorithm. Each input pixel is weighted by three coef- 
ficients generating three partial pixels (K1, K2, K3). The first result accumulator (Q'^j is always valid (Equation 33); the 
second accumulator (Q £ is governed by Equation 34. 



Generating Coefficients 

To build the interpolation into the coefficients, the coefficient reduction step weights the interpolated coefficients 
e, f and c,dby half before combining associated coefficients. 

To reduce the number of possible values an interpolation coefficient can take on we introduce coefficient combi- 
nation terms. Instead of two distinct terms for a and (f/2 + a) we can combine these terms into a single term taking 
advantage of the fact that whenever we would use just the a term, f = 0. Likewise, we can combine b and (b + c/2) 
using just (b + c/2) whenever b is required. 



55 
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CI 



e/2+(a+02) 
eft 



if PS, & PS 3 
otherwise 



(35) 



C2 = 



(b+cft) 

(02+a)+(£+c/2) 
(02 +a) 



if PSl & PS 2 
else if PS, & 7^ 
otherwise 



(3«) 



Ci = 



(**cy2) 

(fl2+a)+(b+cft) 
dft 



if (PB, & P£,) | (PS 3 & PS,) 
else if PS, & PS, 
otherwise 



(37) 



C4 



<02 
0 



if (PS t & PS 2 & PS0 
otherwise 



(38) 



FIGURE 1 3 illustrates the converged coefficient interpolation/reduction block (1301, 1 302). The inputs are the six 
coefficients a - /as described above from circuitry 1003, 1004, the current alignment flags (PB1, PB2, PB3, PS2, PE2, 
and PS3) and the scaling mode (scale down, scale up, pre-interpolation or scale up pre-replication). A series of adders 
1303 forms the required coefficient terms (e.g., b + c,a + V2, etc.) and four multiplexors 1304 select the appropriate 
values for C1 t C2, C3 and C4. The operations of adders 1 303 and multiplexors 1 304 are controlled by the alignment 
flags and the scaling mode from decode circuit 1305, which implements equations 35-38. 

It is not necessary to employ three Z-Engines to generate pre-interpolated coefficients. Note the overlap in FIGURE 
9 of e,/and the previous c,d coefficients, it is sufficient to have two Z-Engines for pre-interpolation with the addition of: 

e,= c M (39) 
fi=cf L1 (40) 

(41) 



PS 3} = PS^ 



FIGURE 4 shows an overview of this two Z-engine architecture. 

The Pre-interpolation Calculation 

In pre-interpolation an input pixel can contribute to up to four output pixels. This requires two pending accumulators 
(A & B) and two result accumulators (Q A & Og). 

Q' A =K1+A (42) 



K2+B 
K2 



if PS 3 
otherwise 



(43) 
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K3 if (PSj | PU,) & (PS, | P£,) 

K2+B else if PS 3 
K2 otherwise 



(44) 
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B2 



-{ 



K4 if (PS, & PS, & PSj) 

AT3 otherwise 
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Va/tfa = 1 



Valid B ={PS 3 I PB,) & (PS, I P^) 
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Extension to Two Dimensions 

The same approach used for scaling down is used to extend scaling up to two dimensions: scaling up is performed 
in two stages, first processing pixels vertically, then passing the vertically weighted pixels on to be scaled horizontally. 
Each dimension has a separate Z-Engine and coefficient reduction complex. The vertical weighting calculation per- 
formed in circuit 1601 (see FIGURE 16) becomes: 

| K9 K10 K11K12] = [C1 y C2 V C3 V C4 V J-P. (48) 

Similarly the horizontal weighting calculation performed in circuit 1602 (see FIGURE 16) becomes: 

[ K1 K2 K3 K4 ) = [ C1 H C2 H C3 H C4 H ] .PV a (49a) 
[ K5K6K7K8) = [C1 H C2 H C3 H C4 H ].PV b (49b) 

In the vertical dimension it is necessary to accumulate two partial pixels for each pixel of an input line, requiring a 
two line cache as wide as the widest Input pixel line. FIGURE 14 diagrams the converged vertical accumulation Im- 
plementation. The inputs are the four partial pixels from the vertical weighting (multipliers) K9 - K12, the two cache 
values which match this input pixel (R1 and R2) and the relevant steering controls derived from the pixel alignment 
flags. Two adders 1402, 1403 are used to generate terms K9+R1 and K10+ R2. Four multiplexors 1404 each select 
from a subset among these six terms to generate two completed vertical partial pixels (PVa and PVb) to pass on to 
the horizontal section and two pending partial pixels (W1 and W2) to store in caches 1603, 1604 for retrieval on the 
next pixel line: 

PV a =K9+R1 (50) 



K10 + R2 
K10 



ifPS 3 
otherwise 



(51) 



45 



50 



Wl 



Kll if (PS 3 | PB X ) & (PS X | PE t ) 

K10 + R2 else if PS 3 
K10 otherwise 



(52) 



55 



11 



EP 0 710 925 A2 



20 



K12 if (PS 3 & PS X & PS 2 ) (53) 

Kll otherwise 



In the horizontal direction, because the vertical calculation can produce two completed pixels, it is necessary to 
70 have two copies of the horizontal accumulation logic. See FIGURE 15 for an overview of the horizontal accumulation 
implementation, which diagrams the two converged horizonal accumulators 1501, 1 502 (one for each of the two vertical 
partial pixel inputs) termed the upper and lower accumulators. Since they are identical, only one will be described The 
inputs to the upper accumulator 1501 are the four partial pixels from the horizontal weighting (multipliers) K1 - K4, the 
two pending accumulators {A and B) and the relevant steering controls derived from the horizontal pixel alignment 
75 flags. Two adders 1503, 1504 are used to generate accumulator terms K1 + A and K2 + B. Four multiplexors 1505 
each select from a subset among these six terms to generate two completed output pixels (Qa and Qb) and two pending 
partial pixels {A and B), which are saved for the next input pixel: 

Q A = K1 + A (54) 



K2 ♦ B if PS, (55) 
K2 otherwise 



25 



30 



A - 



K3 if (PS 3 | PB % ) & (PS % \PE X ) 

else if PS 2 
otherwise 



K2 + B dse if PS, <5*) 



[K4 if (PS 3 & PS, & flSj) (57) 

\ K3 otherwise 

See also FIGURE 1 6 for an overall data flow of the converged architecture, which combines the previous diagrams 
to show an overview of the data flow through the scaling algorithm as detailed in the previous Figures. 

40 Boundary Conditions 

With Pre-lnterpolation we encounter the problem of boundary conditions. On the first pixel of a line and the first 
line of a field the previous pixel or line is not available. To replace this missing data the first pixel and line is logically 
replicated. For the two boundary overlay cases (i & vii) two adjustments need to be made to the algorithm. Because 
45 the inherited e and f coefficients are unknown at the boundary the coefficient interpolator is modified by: 

m if FIRST 

CI = et2+(a+JI2) else if PS % & 1% (58) 
50 \ ef2 otherwise 

and there is no pending data the accumulator is modified by: 

Q (Kl if FIRST (59) 

V* \K1+A otherwise 1 ' 

55 

Because the coefficient combination term (f/2+ a) is included in the boundary cases it is also necessary to initialize 
f = 0. 

A representative hardware environment for practicing the present invention is depicted in FIGURE 17, which illus- 
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trates a typical hardware configuration of a workstation in accordance with the subject invention having a central 
processing unit 10, such as a conventional microprocessor, and a number of other units interconnected via a system 
bus 12. The workstation shown in FIGURE 16 includes a random access memory (RAM) 14, read only memory (ROM) 
16, and input/output (I/O) adapter 18 for connecting peripheral devices such as disc units 20 and tape drives 40 to bus 
12, user interface adapter 22 for connecting keyboard 24, mouse 26, speaker 28, microphone 32, and/or other user 
interface devices such as a touch screen device (not shown) to bus 12, communication adapter 34 for connecting the 
workstation to a data processing network, and display adapter 36 for connecting bus 12 to display device 38. 

The present invention as depicted in FIGURES 6 and 16 may be implemented within adapter 36. 

In practice, the Vertical Coefficient Generator iterates once for each line of a field (the coefficients and pixel flags 
are constant for pixels on that line) and the Horizontal Coefficient Generator iterates once for each pixel on a line. 
Because the Vertical and Horizontal Coefficient Generators are never in operation at the same time, we can use the 
common technique of multiplexing to share a single coefficient generator between the Vertical and Horizontal states. 
For the purposes of this disclosure, the details of this are not relevant. 



Claims 

1 . A method for scaling a video image, comprising the steps of: 

generating coefficients in a vertical direction as a function of an area weighted average of contributions of 
input pixels to output pixels in said vertical direction and as a function of a scaling ratio of said input pixels to 
said output pixels in said vertical direction; 

generating coefficients in a horizontal direction as a function of an area weighted average of contributions of 
input pixels to output pixels in said horizontal direction and as a function of a scaling ratio of said input pixels 
to said output pixels in said horizontal direction; 

multiplying input pixels of said video image by said coefficients generated in said vertical direction as a function 
of an area weighted average of contributions of input pixels to output pixels in said vertical direction and as a 
function of said scaling ratio of said input pixels to said output pixels in said vertical direction, said multiplying 
of said input pixels of said video image by said coefficients generated in said vertical direction resulting in 
vertically scaled pixels; 

multiplying said vertically scaled pixels by said coefficients generated in said horizontal direction as a function 
of an area weighted average of contributions of input pixels to output pixels in said horizontal direction and as 
a function of said scaling ratio of said input pixels to said output pixels in said horizontal direction resulting in 
vertically and horizontally scaled pixels. 

2. The method of claim 1 , wherein said coefficients generated in said vertical direction are also a function of an 
interpolation of said input pixels in said vertical direction. 

3. The method of claim 1 , wherein said coefficients generated in said vertical direction are also a function of a dupli- 
cation of said input pixels in said vertical direction. 

4. The method of claim 1 or 2, wherein said coefficients generated in said horizontal direction are also a function of 
an interpolation of said input pixels in said horizontal direction. 

5. The method of claim 1 or 3, wherein said coefficients generated in said horizontal direction are also a function of 
a duplication of said input pixels in said horizontal direction. 

6. The method of claim 1, wherein said step of generating coefficients in a vertical direction further comprises the 
step of logically doubling said input pixels in said vertical direction. 

7. The method of claim 1 or 5, wherein said step of generating coefficients in a horizontal direction further comprises 
the step of logically doubling said input pixels in said horizontal direction. 

8. The method of claim 1 , wherein said step of generating coefficients in a horizontal direction iterates for each pixel 
on a line of said video image, resetting at horizontal retrace, said step for generating coefficients in a vertical 
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direction iterating for each line of a field, resetting at vertical retrace. 

A system for scaling a video image comprising means for carrying out the steps of the method according to claims 
1to8. 



14 



EP 0710 925 A2 




15 



EP 0710 925 A2 



3 12 2 1 3 











-7/ 


/ 


/ 




P1 


P2 


P3 


P4 






V, 



Q1 Q2 Q3 



Q1=(3P1+1P2)/4 
Q2«(2P2+2P3)/4 
Q3=(1P3+3P4)/4 

FIG. 3 



16 



EP 0710 925 A2 



FIG. 4 



COMPARE 




17 



EP 0710 925 A2 



FIG. 5 




18 



EP 0710 925 A2 




19 



EP 0710 925 A2 




20 



EP 0 710 925 A2 




RESET 



FIG. 10 



rr 



1003, 1004 



INTERPOLATE- 
REPLICATE- 



rrv 



Zout 
C1 



1001 



Z-ENGINE 



C2 
pb 
ps 
pe 



1002 



\ 



Zin 



m 



Zout 



C1 



n C2 

Z-ENGINE 
pb 

ps 
pe 



+SCALE-DOWN/ 
-SCALE-UP 



PIXEL CLOCK 



LATCHES 




d q 

c 



d q 

c 



d q 

c 



d q 

c 



d q 

c 



a 
b 



*pb1 



T)b2 



-pb3 



d q 

c 



tr 



d q 

c 



e 



d q 

c 



d q 

c 



t— ' d 
" f 



d q 

c 



-ps2 



d q 

c 



"ps3 



d q 

c 



*pe2 



21 



EP 0710 925 A2 



FIG. 1 1 



WZZZ& ZZi 



7 



vzzzzzm z. 




V. 




xzzzszzz'A 




22 



EP 0710 925 A2 

FIG. 12 



■ ~ V77777Y Z\~~\ 




23 



EP 0710 925 A2 



FIG. 13 



FROM 
COEFFICIENT 
GENERATION 

a 

FROM b 
1003, c 
1004 d 

e 

f J 



INTER- 
POLATE ' 

REPLICATE - 

pbf 
pb2" 
pb3" 
ps2" 
pe2' 

ps3 • 



ADDERS / 
b+c 
a+e/2+f/2 

a+b+c/2+fy2 
b+c/2 

a+f/2 
a+b+c 



1303 



1304 



C1 
C2 

C3 
C4 
MUX 



DECODE 



1305 



d 


q 


— C1 


c 






d 


q 


— C2 


c 






d 


q 


— *C3 


c 






d 




— ~C4j 


c 





TO 
1601. 
1602 



PIXEL CLOCK 



1301,1302 



COEFFICIENT 
MUX CONTROL 



d q 

c 



d q 

c 



d q 

c 



d q 

c 



VERTICAL ACCUMULA- 
TION MUX CONTROL 

HORIZONTAL ACCUMULA- 
'TION MUX CONTROL 

-CACHE RAM 
READ REQUEST 

-CACHE RAM 
WRITE REQUEST 



PIXEL CLOCK 



24 



EP 0710 925 A2 



FIG. 14 




K11- 
K12- 



1402 
K9+R1 



1403 
K10+R2 



VERTICAL 
ACCUMULATION 
MUX CONTROL 



MUX 



d q 

c 



d q 

c 



d q 

c 



d q 

c 



PIXEL CLOCK 



1404 



PVa 



(TO 



1 

W2 



1401 



25 



EP 0 710 925 A2 



FIG. 15 
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FIG. 16A 
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FIG. 16B 
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